In order to be sure that the operator won't restart processing the custom resource immediately, you could also return the following:

return reconcile.Result{Requeue: true, RequeueAfter: time.Second * 1}, err

Here I'm not able to return reconcile.Result without modifying the signature. It seems to be a helper class and return here seems not good, at least there is no such any method in this class.

Do you think it is OK to send reconcile.Result back though k8s_helper.go#GetEndpointTlsCrt -> create.go#CreateTLSSecret -> che_controller.go#Reconcile ?

Or the following fragment in che_controller would be good enough?

if err := r.CreateTLSSecret(instance, "", "self-signed-certificate"); err != nil {
  return reconcile.Result{Requeue: true, RequeueAfter: time.Second * 1}, err
}

But it would mean that we wait one second on any error, like failed to get namespace, failed to create route, failed to fetch host...
Then why we do not wait 1 second in other places like: when we failed to create service account https://github.com/eclipse/che-operator/blob/9682f3448fe240c216aa22d9d3f56cc056b01494/pkg/controller/che/che_controller.go#L315-L317

BTW Seems that even with return reconcile.Result{}, err there is ~ 1 second between k8s_helper.go#GetEndpointTlsCrt invocations

the first one - is the first invocation
2nd and 3rd are the second invocation.

Here I'm not able to return reconcile.Result without modifying the signature.

Oh, yes, I didn't pay attention to the fact it was inside a utility function.
You might simply add a retryLater boolean return value, and use it in the che_controller reconcile function.

Then why we do not wait 1 second in other places like: when we failed to create service account

Well the existing logic in the controller (that was already there when I started on it) seems to be the following:

• return only an error when it is an unexpected error,
• return a Result (with retry) + error when it's an error that we expect in some scenarios and on which we explicitly want to restart the reconciliation after a while.

Do you think that we should retry if quite unexpected exception occurred like failed to create route?
It's unexpected by maybe k8s/openshift API was temporary unavailable...
WDYT?

I think that if the CreateTLSSecret function returns continueLater with true, then the calling reconcile method should return reconcile.Result{Requeue: true, RequeueAfter: time.Second * 1}, err else it should return nil, err if an unexpected error occured.

yeah, I understand it, let me rephrase my question:
CreateTLSSecret may returns error in different situations:

1. Failed to get an existing self-signed secret.
2. Failed to get existing route (like when API is not available)
3. Failed to create new route (like when API is not available)
4. The fetched route does not have host (it's clear that we should retry in such case)
5. Failed to request route host to retrieve TLS certificate
6. Failed to create a secret with fetched TLS self-signed certificate.

Which of this errors you consider as unexpected and which expected (like 4 about route) ?

To me you would only use continueLater=true in situation 4. We know that in some cases the host can take some time to be added to the route object. It's expected.

Possibly 5 could also be expected (and return continueLater=true, since sometimes the route may take some time to be setup.

But main point is the 4 IMO

Does it seem consistent to you ?

I believe that this PR brings some fix and does not do code base worse.
Your proposed solution totally makes sense but it's more about refactoring the current architecture but not about fixing the issue I initially tried to use.
Sorry, but I'm not able to invest now in operator architecture.

@davidfestal If you agree that this PR is OK to merge - please approve.
If you think that it brings more issues than solves - feel free to close it.